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Program description 



An Ethics Curriculum for Children, a read-aloud literature-based 
curriculum, aims to teach elementary school students seven uni- 
versal attributes of good character. Lessons and home assign- 
ments are organized around multicultural stories. The program 
activities are designed to connect the experiences of characters 



in the stories to students’ own lives. An Ethics Curriculum for 
Children also includes integration of character education themes 
across curricular topics and parental notification and involve- 
ment, optional parts of the implementation of this program. 



Research One study of An Ethics Curriculum for Children met the WWC western Pennsylvania and southern Illinois. The study examined 
evidence standards with reservations. This study included nearly effects on students’ behavior and knowledge, attitudes, and 
1,000 elementary students attending semirural school districts in values.^ 



Effectiveness An Ethics Curriculum for Children was found to have no discernible effects on behavior or on knowledge, attitudes, and values. 



Behavior 

Rating of effectiveness No discernible effects 
Improvement index^ Average: 0 percentile 

points 

Range: -4 to +4 percentile 
points 



Knowledge, attitudes, and values Academic achievement 

No discernible effects Not reported 

Average: +5 percentile points Not reported 

Range: -8 to +14 percentile points 



1. The evidence presented in this report is based on the available research. Findings and conclusions may change as new research becomes available. 

2. These numbers show the average and the range of improvement indices, for aii findings across the two subsamples in the study. The range is based on 
variations in grade ievei. The improvement index for grades 1-3 is -4 and for grades 4-6 is +4 percentiie points. 
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Developer and contact 

Heartwood Institute, 425 North Craig Street, Suite 302, 
Pittsburgh, PA 15213. Web: www.heartwoodethics.org. Emaii: 
hrtwood@heartwoodethics.org. Teiephone: 800-432-7810. 

Scope of use 

An Ethics Curriculum for Children, created in the early 1990s, 
has been used in more than 1,500 schools in 40 states. Infor- 
mation is not available about the characteristics of students, 
schools, or districts using the curriculum. An Ethics Curriculum 
for Children may have changed since the study was conducted. 
The WWC recommends asking the developer for information 
about the most current version of this curriculum and taking into 
account that student demographics and school context may 
affect outcomes. 

Teaching 

Each of the six elementary curriculum kits includes 14 les- 
sons, with two lessons for each of the seven core attributes. 
Each lesson includes reading an illustrative story aloud, class 
discussion, reflective journaling, and small-group or individual 
activities. The developers suggest creating a comfortable story 
corner for children in the classroom. They estimate that two to 
four weeks are needed for children to assimilate each attribute. 
Teachers are encouraged to extend the lessons from the stories 
to other subject matter. The curriculum manual offers ideas for 
planning lessons and integrating character education-based 



activities across academic areas. Teachers can engage parents 
and families through family orientation meetings or by sending a 
letter home that explains the program. 

Curriculum kits provide the materials to implement the curric- 
ulum. A Spanish kit with Spanish language materials is available 
for use in kindergarten through grade three. Supplemental mate- 
rials — such as posters, maps, buttons, and music CDs — can 
be obtained through the Heartwood Institute’s website at www. 
heartwoodethics.org/. The Institute offers teaching manuals, 
teaching videos, a do-it-yourself training module that includes 
videos, and both a half-day and a full-day on-site professional 
development workshop led by a Heartwood staff trainer. A help 
desk, staffed by teachers with experience implementing the cur- 
riculum in the classroom, is available three days a week. 

Cost 

Curriculum kits are sold in sets that cover two grades per set 
(PreK/K, K/1, 1/2, 2/3, 3/4, 4/5, and 5/6) and cost $438 plus 
a 9% shipping and handling charge. Kits for grades 4-6 cost 
$448 plus shipping and handling. The Spanish kit (grades K-3) 
costs $495. Various training workshops are available, including 
half-day and full-day workshops ranging from $750 to $2,000. 
Do-It-Yourself Training for Teachers ($29.95) provides a scripted 
training program with instructions. The book A Lesson in Char- 
acter, designed as a primer for teaching An Ethics Curriculum for 
Children, is available for $9.95 per grade. 



Three studies reviewed by the WWC investigated the effects of 
An Ethics Curriculum for Children. One study (Leming, 2000) was 
a quasi-experimental design that met WWC evidence standards 
with reservations. The other two studies did not meet WWC 
evidence screens. The Leming study included 965 elementary 
school students in semi-rural school districts in western 



Pennsylvania and southern Illinois. It compared outcomes for 
students participating in An Ethics Curriculum for Children pro- 
gram with outcomes for students in comparison classes that did 
not use a character education curriculum. The study focused on 
An Ethics Curriculum for Children as implemented in classrooms 
rather than as a schoolwide intervention. 
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Effectiveness 



Findings 

The WWC review of character education addresses student 
outcomes in three domains: behavior; knowiedge, attitudes, and 
vaiues; and academic achievement. 

Behavior. On one behaviorai measure, ethical conduct, the 
study reported a statistically significant worse outcome for 
students in grades 1-3 using An Ethics Curriculum for Children 
(compared with the group that did not receive the curricuium) 
and a statisticaiiy significant better outcome for the students in 
grades 4-6 using An Ethics Curriculum for Children. However, 
the differences for students in grades 1-3 and for students in 
grades 4-6 (as caicuiated by the WWC) were neither statisticaiiy 
significant nor iarge enough to be considered substantively 
important using WWC criteria.^ On a second behaviorai mea- 
sure (simulated cheating), there were no statistically significant 
or substantiveiy important differences for either grade range. 
Another behaviorai measure, schooi discipiinary reports, was 
also included in the study, but the results were presented only at 
the aggregate school level and could not be compared in a way 
that met WWC evidence standards. So this measure was not 
included in the review. 



Knowledge, attitudes, and values. For grades 1-3, Leming 
reported statistically significant better outcomes for students 
using An Ethics Curriculum for Children on two character- 
related measures (ethical understanding and ethnocentrism) and 
no statistically significant difference on a third (ethical sensibil- 
ity). For grades 4-6 statistically significant better outcomes were 
reported for students using An Ethics Curriculum for Children 
on one measure (ethical understanding), a statistically significant 
worse outcome on a second (ethical sensibility), and no differ- 
ence on the third (ethnocentrism). The differences (as calculated 
by the WWC) in this domain were neither statistically significant 
nor large enough to be considered substantively important.® 

Rating of effectiveness 

The WWC rates interventions as positive, potentially positive, 
mixed, no discernible effects, potentially negative, or negative. The 
rating of effectiveness takes into account four factors: the quality 
of the research design, the statistical significance of the findings 
(as calculated by the WWC), the size of the difference between 
participants in the intervention and the comparison conditions, and 
the consistency in findings across studies (see the WWC Intervention 
Rating Scheme) . 



The WWC found >lii Ethics 
Curriculum for Children 
to have no discernible 
effects on behavior 
or on knowledge, 
attitudes, and values 



Improvement index 

For each outcome domain, the WWC computed an improvement 
index based on the average effect size (see the WWC Improve- 
ment Index Technical Paper) . The improvement index represents 
the difference between the percentile rank of the average student 
in the intervention condition and the percentile rank of the average 
student in the comparison condition. Unlike the rating of effective- 
ness, the improvement index is based entirely on the size of the 
effect regardless of the statistical significance of the effect, study 
design, or analysis. The improvement index can take on values 
between -50 and +50, with positive numbers denoting favorable 
results. The average improvement index for behavior is 0 percentile 
points with a range of -4 to +4 percentile points across findings. 
The average improvement index for knowledge, attitudes, and 
values is 5 percentile points, with a range of -8 to +14 percentile 
points across findings. 



Summary 

The WWC reviewed three studies on An Ethics Curriculum 
for Children. One study met WWC evidence standards with 
reservations. It reported student outcomes in the behavior 
and attitudes, knowledge, and values domains for students 
in grades 1-3 and students in grades 4-6 separately. None 
of the WWC-computed overall effect sizes were statistically 
significant or large enough to be substantively important. 

So the WWC rated the program as having no discernible 
effects in the behavior or the knowledge, attitudes, and values 
domains. Character education, an evolving field, is beginning 
to establish a research base. The evidence presented in this 
report is limited and may change as new research emerges. 



3. The level of statistical significance was calculated by the WWC and where necessary, corrects for clustering within classrooms or schools, and for 
multiple comparisons. For an explanation see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to 
calculate the statistical significance. In the case of the Ethics Curriculum report, corrections for clustering and multiple comparisons were needed. 
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ReferenCGS Met WWC evidence standards with reservations 

Leming, J. S. (2000). Tell me a story: An evaluation of a litera- 
ture-based character education programme. Journal of Moral 
Education, 29, 413-427. 

Additional citations 

Leming, J., Henricks-Smith, A., & Antis, J. (1997). An evalua- 
tion of the Heartwood Institute’s An Ethics Curriculum for 
Children: Final report. Paper presented at the annual meet- 
ing of the American Educational Research Association, 
Chicago, IL. (Revised 4/1/2000). 

Leming, J. S., & Silva, D. Y. (2001). A five year follow-up 
evaluation of the Heartwood Ethics curriculum on the 



development of children’s character. Retrieved from 

http://www.heartwoodethics.org/docs/Heartwood_5_ 

Year_Report.pdf 

Did not meet WWC evidence screens 

Buttram, J. L., Kruse, J., & Silder, J. (1992). Evaluation of the 
Heartwood Program: Final report. Philadelphia, PA: Research 
for Better Schools, Inc."^ 

Piscolish, M. (1991). Heartwood Program pilot year evaluation: 
1990-91 full report. Pittsburgh, PA: Pittsburgh Board of 
Public Education.® 



For more information about specific studies and WWC caicuiations, piease see the WWC Ethics Curricuium 
Technicai Appendices . 



4. Did not meet WWC evidence screens because study sample is not relevant; it did not assess student outcomes. 

5. Did not meet WWC evidence screens because study did not use a comparison group. 
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Appendix 



Appendix A1 


Study characteristics: Leming, 2000 (quasi-experimentai design) 


Characteristic 


Description 


Study citation 


Leming, J. S. (2000). Tell me a story: An evaluation of a literature-based character education programme. Journal of Moral Education, 29, 413-427. 


Participants 


The study Included 965 students in grades 1-6 from four schools in two school districts. The student popuiation in one school district was 95.6% Caucasian, with 25% 
receiving free or reduced-price iunch. The student popuiation in the two participating schools in the other school district was 85.6% Caucasian, with 50% receiving free or 
reduced-price lunch. 


Setting 


Two participating schools (one intervention, one comparison) were in a mid-sized, semirural school district in western Pennsylvania; the other two participating schools (one 
intervention, one comparison) were in a small, semirural school district in southern Illinois. The comparison students were drawn from matched schools in the same school 
districts as the intervention students. 


Intervention 


The intervention group participated in Heartwood Ethics Curriculum for Children for the course of an academic school year. The literature-based program used folktales, 
folklore, and fairy tales to teach seven core values. Lessons included an opening discussion, story, discussion questions, reinforcement activity, and writing about the concept. 


Comparison 


No information was provided on comparison students other than they did not receive Heartwood Ethics Curriculum for Children. 


Primary outcomes 
and measurement 


The study-specific measures addressed understanding of the curriculum’s seven core values (ethical understanding), preferences for exemplifying core values (ethical sen- 
sibility), behavior related to core values (ethical conduct), and affinity for ethnically diverse children (ethnocentrism). The measures were developed for this study to measure 
specific outcomes. (See Appendices A2.1 and A2.2.) 


Teacher training 


Teachers implementing Heartwood Ethics Curriculum for Children attended a half-day training course before the start of the school year. 



WWC Intervention Report 






Appendix A2.1 Outcome measures in the behavior domain 



Outcome measure 


Description 


Ethical conduct 


Ratings by teachers of character-related behavior for individuai students. This measure uses a five-point Likert-type scale developed by the author (as cited in Leming, 
2000). 



Appendix A2.2 Outcome measures in the knowiedge, attitudes, and vaiues domain 



Outcome measure 


Description 


Ethical understanding 


Responses to questionnaire items developed by the author about the meaning of the currioulum’s seven core attributes: courage, honesty, hope, justioe, love, loyalty, and 
respect. Grades 1-3 used 14 items in multiple choice format (two possible answers); grades 4-6 used a sentence completion format for testing understanding of the 
seven attributes (as cited in Leming, 2000). 


Ethical sensibility 


Responses to questionnaire items developed by the author that asked about agreement with or valuing of the curriculum's seven core attributes: courage, honesty, hope, 
justioe, love, loyalty, and respeot. Grades 1-3 used a test of the student's agreement with 20 “1 would” statements (e.g., “1 would tell them to stop”) with three levels of 
response (yes, no, not sure). Grades 4-6 used the same 20 items with a five-level Likert-type soale (as cited in Leming, 2000). 


Ethnocentrism 


Ratings of social distance, measured by photographs of ethnioally diverse ohildren for whether the student would like to have them as a friend; used with white students 
only. This measure was adapted from the Social Distance Measure (as cited in Leming, 2000). 
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Appendix A3.1 Summary of study findings inciuded in the rating for the behavior domain^ 









Author’s findings from the study 














Mean outcome'’ 
(standard deviation”) 




WWC calculations 




Outcome measure'^ 


Study 

sample 


Sample size 
(classrooms/ 
students) 


Ethics 

Curriculum for 
Children group 
(column 1) 


Comparison 
group 
(column 2) 


Mean difference” 
(column 1- 
column 2) 


Statistical 
significance^ 
Effect size' (at a = 0.05) 


Improvement 

index'' 



Leming, 2000 (quasi-experimental design) 



Ethical conduct 


Grades 1-3 


4/485 


67.93 

(11.54) 


69.09 

(14.10) 


-1.16 


-0.09 


ns 


-4 


Average' for behavior (Leming, 2000: Grades 1-3) 










-0.09 


ns 


-4 


Ethical conduct 


Grades 4-6 


4/385 


64.48 

(12.79) 


63.20 

(12.54) 


-Hi. 28 


0.10 


ns 


-f4 


Average' for behavior (Leming, 2000: Grades 4-6) 










0.10 


ns 


-f4 


Domain average' for behavior across ali studies 










0.01 




0 



ns=not statistically significant 



a. This appendix reports findings considered for the effectiveness rating and the improvement index. 

b. The study author adjusted the means for pretest differences between groups. 

c. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

d. The study author analyzed two additional outcomes in this domain (cheating on drawing task grades 1-3, cheating on drawing task grades 4-6) but did not provide means or standard deviations, so they are not listed in this table. None 
of the three outcomes was statistically significant. 

e. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

f. For an explanation of the effect size calculation, please see the WWC Technical Working Paper on Effect Size . 

g. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. The level of statistical significance was calculated by the WWC and where necessary, cor- 
rects for clustering within classrooms or schools, and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to calculate the statisti- 
cal significance. In the case of the Ethics Curriculum report, corrections for clustering and multiple comparisons were needed. 

h. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and -t-50, with positive numbers denoting favorable results. 

i. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average improvement indices are calculated from the average effect sizes. 
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Appendix A3.2 Summary of study findings inciuded in the rating for the knowiedge, attitudes, and vaiues domain^ 



Author’s findings from the study 



Mean outcome'’ 

(standard deviation”) WWC calculations 









Ethics 
















Sample size 


Curriculum for 


Comparison 


Mean difference” 




Statistical 






Study 


(classrooms/ 


Children group 


group 


(column 1- 




significance^ 


Improvement 


Outcome measure'' 


sample 


students) 


(column 1) 


(column 2) 


column 2) 


Effect size' 


(at a = 0.05) 


index'' 




Leming, 2000 (quasi-experimental design) 


Ethical understanding 


Grades 1-3 


4/482 


23.35 

(2.01) 


22.75 

(1.93) 


0.60 


0.31 


ns 


-Hi 2 


Ethnocentrism 


Grades 1-3 


4/447 


28.10 

(24.58) 


33.97 

(25.54) 


5.87 


0.23 


ns 


-f9 


Ethical sensibility 


Grades 1-3 


4/479 


51.84 

(6.87) 


52.18 

(6.34) 


-0.34 


-0.05 


ns 


-2 


Average' for knowledge, attitudes, values (Leming, 2000: Grades 1-3) 








0.16 


ns 


-f6 


Ethical sensibility 


Grades 4-6 


4/369 


75.09 

(11.57) 


77.30 

(10.30) 


2.21 


0.20 


ns 


-f14 


Ethical understanding 


Grades 4-6 


4/376 


14.41 

(2.44) 


13.54 

(2.57) 


-0.87 


-0.35 


ns 


-8 


Average' for knowledge, attitudes, values (Leming, 2000: Grades 4-6) 








0.07 


ns 


-f3 


Domain average' for knowledge, attitudes, values across all studies 








0.12 




-f5 



ns = not statistically significant 



a. This appendix reports findings considered for the effectiveness rating and the improvement index. 

b. The study author adjusted the means for pretest differences between groups except for one outcome (ethical understanding for grades 4-6). 

c. The standard deviation across all students in each group shows how dispersed the participants' outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

d. The study author analyzed one additional outcome in this domain (ethnocentrism grades 4-6) but did not provide means or standard deviations, so it is not listed in this table. The outcome was not statistically significant. 

e. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. The sign of the mean difference for ethnocentrism scores was reversed because greater ethnocentrism 
was considered a negative outcome. 

f. For an explanation of the effect size calculation, please see the WWC Technical Working Paper on Effect Size . 

g. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. The level of statistical significance was calculated by the WWC and where necessary, cor- 
rects for clustering within classrooms or schools, and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to calculate the statisti- 
cal significance. In the case of the Ethics Curriculum report, corrections for clustering and multiple comparisons were needed. 

h. The Improvement Index represents the difference between the percentile rank of the average student In the Intervention condition and that of the average student In the comparison condition. The Improvement Index can take on values 
between -50 and -t-50, with positive numbers denoting favorable results. 

i. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average Improvement Indices are calculated from the average effect sizes. 
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Appendix A4.1 Rating for the behavior domain 

The WWC rates interventions as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative.^ 

For the outcome domain of behavior, the WWC rated Heartwood Ethics Curriculum for Children as having no discernible effects. It did not meet the criteria for 
positive effects, because it only had one study. In addition, it did not meet the criteria for other ratings (potentially positive effects, mixed effects, potentially negative 
effects, and negative effects) because none of the studies showed statistically significant or substantively important effects. 

Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: None of the studies shows a statistically significant or substantively important effects, either positive or negative. 

Met. The WWC analysis found no statistically significant or substantively important effects in this domain. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. The WWC analysis found no statistically significant positive effects in this domain. Heartwood Ethics Curriculum for Children had only 
one evaluation study meeting WWC evidence standards that reported findings on behavior, and so did not meet this criterion. Further, that study 
did not meet WWC evidence standards for a strong design. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. The WWC analysis found no statistically significant or substantively important negative effects in this domain. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, thus qualifying as a positive effect. 

Not met. The WWC analysis found no statistically significant or substantively important positive effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect. Fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Not met. No studies showed a statistically significant or substantively important negative effect. Because one study showed indeterminate effects 
and no studies showed statistically significant or substantively important positive effects, Heartwood Ethics Curriculum for Children did not meet 
this criterion. 

(continued) 
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Appendix A4.1 Rating for the behavior domain (continued) 



Mixed effects: Evidence of both positive and negative effects. 

• Criterion 1: At ieast one study showing a statisticaiiy significant or substantiveiy important positive effect. At ieast one study showing a statisticaiiy significant or 
substantiveiy important negative effect, but no more such studies than the number showing a statisticaiiy significant or substantiveiy important positive effect. 

Not met. The WWC anaiysis found no statisticaiiy significant or substantively important positive or negative effects in this domain. 

• Criterion 2: At ieast one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statisticaiiy significant or substantiveiy important effect. 

Not met. The WWC analysis found no statisticaiiy significant or substantively important effects in this domain. 

Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statisticaiiy significant or substantively important negative effect. No studies showing a statisticaiiy significant or substan- 
tiveiy important positive effect, or more studies showing statistically significant or substantively important negative effects than showing statisticaiiy significant or 
substantively important positive effects. 

Not met. The WWC analysis found no statistically significant or substantively important negative effects in this domain. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which is based on a strong design. 

Not met. The WWC analysis found no statistically significant negative effects in this domain. 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. The WWC analysis found no statistically significant or substantively important positive effects in this domain. 



1 . For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effect. The WWC also considers the size of the domain level effect for ratings of 
potentially positive effects. See the WWC Intervention Rating Scheme for a complete description. 
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Appendix A4.2 Rating for the knowiedge, attitudes, and vaiues domain 

The WWC rates interventions as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative.^ 

For the outcome domain of knowledge, attitudes, and values, the WWC rated An Ethics Curriculum for Children as having no discernible effects. It did not meet the 
criteria for positive effects, because it only had one study. In addition, it did not meet the criteria for other ratings (potentially positive effects, mixed effects, potentially 
negative effects, and negative effects) because none of the studies showed statistically significant or substantively important effects. ^ 

Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: None of the studies shows a statistically significant or substantively important effect, either positive or negative. 

Met. The WWC analysis found no statistically significant or substantively important effects in this domain. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. The WWC analysis found no statistically significant positive effects in this domain. An Ethics Curriculum for Children had only one evalua- 
tion study meeting WWC evidence standards that reported findings on knowledge, attitudes, and values, and so did not meet this criterion. Further, 
that study did not meet WWC evidence standards for a strong design. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. The WWC analysis found no statistically significant or substantively important negative effects in this domain. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, thus qualifying as a positive effect. 

Not met. The WWC analysis found no statistically significant or substantively important positive effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect. Fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Not met. No studies showed a statistically significant or substantively important negative effect. Because one study showed indeterminate effects 
and no studies showed statistically significant or substantively important positive effects. An Ethics Curriculum for Children did not meet this 
criterion. 

(continued) 
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Appendix A4.2 Rating for the knowiedge, attitudes, and vaiues domain (continued) 



Mixed effects: Evidence of both positive and negative effects. 

• Criterion 1: At ieast one study showing a statisticaiiy significant or substantiveiy important positive effect. At ieast one study showing a statisticaiiy significant or 
substantiveiy important negative effect, but no more such studies than the number showing a statisticaiiy significant or substantiveiy important positive effect. 

Not met. The WWC anaiysis found no statisticaiiy significant or substantively important positive or negative effects in this domain. 

• Criterion 2: At ieast one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statisticaiiy significant or substantiveiy important effect. 

Not met. The WWC analysis found no statisticaiiy significant or substantively important effects in this domain. 

Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statisticaiiy significant or substantively negative effect. No studies showing a statistically significant or substantively 
important pos/'f/Ve effect, or more studies showing statistically significant or substantively negative effects than showing statistically significant or substantively 
important positive effects. 

Not met. The WWC analysis found no statistically significant or substantively negative effects in this domain. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which is based on a strong design. 

Not met. The WWC analysis found no statistically significant negative effects in this domain. 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. The WWC analysis found no statistically significant or substantively important positive effects in this domain. 



1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effect. The WWC also considers the size of the domain level effect for ratings of 
potentially positive effects. See the WWC Intervention Rating Scheme for a complete description. 
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